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I. 


INTRODUCTION 


A.  GENERAL  BACKGROUND 

As  the  21st  century  approaches,  the  Government  is  reforming  the  national  health 
care  system.  The  goals  of  the  reform  initiative  begin  with  ‘inanaged  care”  and  better 
access  to  health  care  for  citizens.  Striving  to  meet  these  goals  and  incorporating  many 
lessons  learned  in  the  1980s,  the  Department  of  Defense  created  the  ‘Gateway  to  Care 
Program.”  The  goal  of  the  program  was  to  give  hospital  commanders  more  power  to 
treat  military  health  care  beneficiaries  by  creating  local  physician  networks  to  meet  the 
needs  of  the  supported  population.  Health  care  in  the  military  has  experienced  many 
changes  in  the  1990s  and  will  continue  to  change  even  more  rapidly  in  the  remaining  years 
of  the  decade.  The  Department  of  Defense  can  provide  a  testbed  to  reengineer  the  health 
care  system  to  provide  greater  access  to  care,  improved  productivity,  increased  efficiency, 
equity,  convenience,  and  reduced  costs. 

The  reengineering  process  begins  by  inserting  information/communication 
technology  into  the  military  health  care  system.  The  latest  initiative  in  support  of  that  goal 
is  called  ‘Telemedicine.”  General  Gordon  R.  Sullivan,  Chief  of  Staff  of  the  Army,  stated 
in  a  recent  National  Forum  on  Telemedicine  that  ‘We  project  power  by  leveraging 
telemedicine  on  the  battlefield.”  In  this  sense,  telemedicine  reduces  risk  to  Service 
members  and  uses  technology  to  maintain  the  edge  over  hostile  and  threat  forces. 

In  September  1994,  the  Assistant  Secretary  of  Defense,  Dr.  Stephen  C.  Joseph, 
charged  the  Army  Surgeon  General,  Lieutenant  General  Alcide  LaNoue  with  responsibility 
for  establishing  a  plan  to  develop  telemedicine  projects  to  support  the  National  Health 
Care  Reform  goals  and  objectives.  The  vision  and  over-arching  goals  of  the  testbed 
organization  are  shown  in  Figure  1-1. 
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Telemedicine  Principles 

Over-arching  Goals: 

•  Readiness  Orientation 

•  Patient  Focus 

•  Rapid  Prototyping 

•  Outcome-based  Metrics 

•  Open  Systems  Architecture 

•  Sound  Business  Practices 

projects  world 

class  health  care  to  patients  and 
providers  anytime,  anywhere. 


Figure  1-1.  Telemedicine  Vision  and  Over-arching  Goals 

The  Medical  Digital  Imaging  Support  system  is  a  project  under  the  telemedicine 
umbrella  that  digitizes  patient  images  and  sends  them  via  fiber  optic  cable  to  a  storage 
unit.  The  computer  network  links  different  types  of  imaging  equipment  or  modalities 
within  a  facility  including:  computed  radiography,  computed  topography,  digital 
angiography,  nuclear  medicine,  ultrasound,  magnetic  resonance  and  digital  spot  imaging 
(Smith,  1994).  Images  are  stored  in  a  work  storage  unit  and  archived  images  are  further 
stored  within  an  optical  disk  jukebox  (See  Figure  1-2).  There  are  three  levels  of  data 
storage:  the  short  term  which  stores  images  in  the  system’s  random  access  memory, 
intermediate  storage  which  stores  the  images  on  magnetic  media,  and  long  term  storage 
which  stores  images  on  optical  disc.  The  system  allows  the  physician  to  access  stored 
images  from  the  worksite.  With  film-based  imaging  technology,  physicians  would  have  to 
retrieve  images  from  a  storage  area  often  in  a  remote  part  of  the  hospital.  The  system  also 
has  a  wider  range  of  exposure  times,  which  makes  the  system  more  forgiving  in  producing 
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readable  images  for  the  inexperienced  technician.  The  implication  is  that  less  images  are 
necessary  and  the  system  may  reduce  image  retakes  necessary  with  film-based  technology. 

In  the  future,  after  networks  are  installed  within  a  facility,  they  can  be  linked  to 
other  satellite  facilities  using  “teleradiology.”  The  teleradiology  capability  allows  the 
medical  treatment  facility  (MTF)  to  communicate  with  other  MTFs  in  a  network  via  fiber 
optic  cable  or  satellite  (Cade,  1993). 


Image  Acquisition  Subsystem 


Figure  1-2.  The  MDIS  System  Configuration 
The  first  Department  of  Defense  (DoD)  facility  to  receive  the  MDIS  system  was 
Madigan  Army  Medical  Center  (MAMC).  Madigan  is  installing  the  3rd  phase  of  a  3  phase 
MDIS  implementation  schedule.  As  with  most  new  technology  systems,  there  is  a  learning 
curve  associated  with  using  the  system  and  the  first  operational  users  experience  growing 
pains  as  they  learn  to  use  the  system  effectively.  Several  modifications  to  the  software 
interfaces  between  hospital  information  systems  and  between  modalities  has  slowed  the 
full  implementation  of  the  MDIS  system  at  Madigan. 
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The  MDIS  Project  Manager  has  a  vision  to  modernize  hospital  imaging  systems  to 
meet  the  current  and  future  needs  of  DoD  medical  treatment  facilities.  In  support  of  this 
vision,  Lieutenant  General  Frank  F.  Ledford,  the  previous  Army  Surgeon  General  said,  “I 
am  totally  committed  to  making  imaging  a  reality  for  military  medicine  in  partnership  with 
our  sister  services.  We  have  a  chance  to  design  the  future  and  I  intend  to  grasp  it”  (Cade, 
1993).  Georgetown  University  and  University  of  Washington  conducted  a  Digital 
Imaging  Network  System  (DINS)  evaluation  project  which  provided  a  benchmark  in 
picture  archiving  and  communications  technology.  There  were  four  major  conclusions  of 
the  project  as  follows: 

•  Commercially  available  digital  technology,  used  under  appropriate  management 
and  with  continuous  vendor  support,  can  perform  the  tasks  necessary  to 
maintain  clinical  utility. 

•  It  was  possible  to  communicate,  archive,  and  display  images  of  diagnostic 
quality  on  a  computer  monitor. 

•  The  technology  could  improve  physician  efficiency  and  availability  to  treat 
more  patients. 

•  The  technology  was  accepted  by  a  diverse  group  of  referring  physicians  and 
radiologists.  The  group  included  neuroradiology,  gastrointestinal,  nuclear 
medicine,  the  intensive  care  nursery,  and  radiation  medicine.  (Crowe,  1991) 

Soon  after  the  evaluation  project  was  completed,  in  September  1991,  the 
Government  awarded  Loral  a  $209  million  indefinite  requirements  contract  to  develop, 
test,  and  field  MDIS  to  DoD  medical  treatment  facilities  (Cade,  1993).  To  date,  the 
Project  Office  has  issued  1 1  delivery  orders  for  Army  and  Air  Force  medical  treatment 
facilities  against  the  contract  which  expires  in  September  1995. 

Seahawk  is  a  teleradiology  project  in  the  Northwest  region  that  will  link  several 
medical  facilities  in  a  joint  Service  imaging  network  called  the  U.S.  West  Lion  Fiber  Ring. 
The  network  consists  of  Army,  Air  Force,  Navy,  and  Veterans  Administration  medical 
treatment  facilities  and  the  University  of  Washington  (See  Figure  1-3). 


4 


U.S.  West  Lion  Fiber  Ring 

American  Lake  VA  Hospital 

^  Kfffl  Seattle  V  A  Hospital 

US  AF  McChord 

Everett  Naval  Hospital 

|  Spokes _ 

Figure  1-3.  Project  Seahawk  Network  Configuration 

The  project  received  a  $10  million  appropriation  in  1994  and  the  equipment  and 
installation  is  expected  to  cost  $6.5  million.  The  project  is  designed  to  increase  efficiency 
through  live  consultations  via  videoteleconferencing  and  decrease  patient  referrals  to  other 
facilities.  Cost  savings  may  result  by  having  radiologists  consult  via  telemedicine, 
eliminating  contracted  radiologists.  The  project  is  also  expected  to  improve  the  quality  of 
patient  care  by  providing  faster  patient  services. 


University  of  Washington 


B.  PURPOSE 

The  purpose  of  this  research  is  to  conduct  a  cost  effectiveness  analysis  comparing 
the  two  types  of  imaging  technologies  used  by  two  major  Army  Medical  Centers.  Much  of 
this  research  will  build  on  previous  studies,  which  captured  the  costs  and  benefits  of 
digital  imaging  technology.  Ideas  for  the  research  project  came  from  the  “Futures” 
section  of  a  preliminary  payback  study  conducted  by  a  Government  contractor.  The 
Futures  section  presents  ideas  about  potential  cost  savings  from  the  system.  These 
savings  need  to  be  measured,  validated,  and  documented. 

The  three  areas  covered  in  detail  in  this  thesis  include  the  amount  of  images  that 
require  retakes,  number  of  lost  images,  and  time  it  takes  for  a  physician  to  retrieve  an 
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image  taken  during  previous  patient  visits.  Cost  effectiveness  is  measured  by  the 
technology’s  effectiveness  relative  to  the  dollars  invested. 

C.  RESEARCH  QUESTIONS 

The  primary  research  question  is:  “Are  there  cost  effectiveness  differences  between 
the  state-of-the-art  digital  imaging  system  and  the  traditional  film-based  imaging 
technology  used  most  frequently  by  Government  and  civilian  medical  treatment  facilities?” 
This  research  will  address  the  following  subsidiary  research  questions: 

1 .  Does  digital  imaging  reduce  the  amount  of  retakes  compared  to  traditional  film- 

based  systems? 

2.  Does  digital  imaging  reduce  the  number  of  images  lost  within  a  medical 

treatment  facility? 

3.  Does  digital  imaging  improve  physician  efficiency  by  reducing  the  time  to 

retrieve  images? 

4.  What  is  the  effectiveness  per  investment  dollar  for  both  systems? 

D.  SCOPE,  LIMITATIONS,  AND  ASSUMPTIONS 

The  emerging  digital  technology  has  the  potential  of  replacing  existing  film-based 
technology  in  civilian  as  well  as  military  medical  treatment  facilities.  Therefore,  it  is 
paramount  that  military  leaders  make  informed  decisions  about  whether  to  purchase  new 
technology.  Compared  to  film-based  technology,  scientists  and  medical  professionals  have 
discovered  many  applications  of  digital  imaging  technology  which  improve  the  quality  of 
patient  care  as  well  as  efficiency.  For  example,  teleradiology  allows  remote  diagnostic 
consultation  and  image  transfer  support  not  possible  under  the  old  system.  With  improved 
access  to  historical  patient  images,  teaching  files  can  also  be  constructed  to  train 
radiologists  and  clinicians  to  detect  disease  and  illnesses  through  imaging  technology. 

This  research  focuses  on  the  clinician,  who  is  the  primary  user  of  the  MDIS 
system.  The  specific  areas  addressed  include  image  retake  rates,  frequency  of  lost  images, 
and  time  to  retrieve  archival  images.  Collected  data  are  analyzed  both  quantitatively  and 
qualitatively  using  questionnaires  and  interviews.  Since  project  Seahawk  is  still  in  the 
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design  phase,  the  cost  effectiveness  measure  used  for  this  project  is  relatively  limited.  The 
MDIS  system  is  functional  within  Madigan,  thus  only  operations  used  currently  are 
included  in  this  project.  The  teleradiology  applications  are  excluded  from  this  study. 

A  major  limitation  of  this  study  is  that  data  are  only  collected  at  two  sites.  Since 
Madigan  is  the  first  facility  to  receive  the  MDIS  system,  and  has  the  most  experience  with 
digital  imaging,  it  will  be  the  site  for  MDIS  data  collection.  The  representative  major  film- 
based  medical  facility  is  Fitzsimons.  The  digital  imaging  implementation  is  not  yet 
complete  at  Madigan.  Therefore,  the  results  of  this  research  only  represent  a  portion  of 
the  total  impact  digital  imaging  technology  has  on  a  facility. 

The  primary  assumption  made  in  this  project  is  that  the  two  sites  chosen  have 
similar  missions,  supported  populations,  radiology  protocols,  and  technical  expertise. 
Another  assumption  is  that  physician  questionnaires  at  each  site  do  in  fact  measure  how 
the  imaging  technology  is  perceived  by  the  system’s  primary  user. 

E.  METHODOLOGY  OVERVIEW 

The  first  step  in  obtaining  information  for  this  paper  is  a  literature  search  on  the 
MDIS  system,  cost  effectiveness  analysis  techniques,  and  technical  papers  about  MDIS 
and  other  Picture  Archiving  and  Communications  (PACS)  systems.  The  next  step  is  to 
collect  data  at  two  comparable  major  medical  treatment  facilities.  Madigan  and 
Fitzsimons  are  both  Army  regional  medical  facilities,  teaching  hospitals  for  military 
physicians,  and  support  a  mixed  population  of  active  duty  military,  dependents,  and 
retirees.  Data  were  collected  using  two  techniques:  questionnaires,  and  interviews  with 
the  hospital  staff.  Personal  interviews  were  conducted  with  key  staff  at  Madigan  and 
Fitzsimons.  The  information  collected  in  the  interviews  supplements  and  verifies  the 
questionnaire  data  to  validate  the  results.  Several  administrative  areas  of  each  facility 
provided  cost  and  workload  data.  The  administrative  areas  from  which  data  were 
collected  include:  resource  management,  patient  administration,  logistics,  radiology,  and 
legal  staff. 
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F.  ORGANIZATION  OF  THE  STUDY 

This  thesis  is  divided  into  five  chapters,  beginning  with  the  introduction  as  the  first 
chapter.  Chapter  II  provides  background  information  on  previous  economic  studies  on 
the  MDIS  system.  Chapter  III  discusses  the  methodology  used  to  gather  and  analyze  the 
data.  Chapter  IV  is  a  detailed  analysis  of  the  data.  Answers  to  each  of  the  subsidiary 
research  questions  are  provided  within  this  chapter.  Chapter  V  summarizes  the  results  and 
provides  insight  to  decision  makers  about  the  effectiveness  of  each  system  in  terms  of 
dollars  invested.  This  chapter  also  includes  recommendations  for  future  studies 
concerning  the  cost  effectiveness  of  digital  imaging  technology.  The  Appendices  include 
the  questionnaire  instrument,  detailed  questionnaire  results,  and  a  format  for  interviews. 
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H.  BACKGROUND 


The  world  of  radiographic  technology  has  undergone  tremendous  evolutionary 
change  within  the  last  century.  One  radiology  technologist  with  30  years  experience  in  the 
field  describes  this  evolution  as  “glass  to  glass.”  Exactly  100  years  ago  a  German 
physicist  named  Wilhelm  Konrad  Roentgen  discovered  the  X-ray  for  use  in  diagnostic 
imaging.  In  the  early  days,  diagnostic  images  were  captured  on  photographic  glass  plates 
covered  with  a  thick  emulsion.  After  an  image  was  taken,  the  emulsion  was  scraped  off 
and  the  glass  plates  were  used  again.  Today  these  images  are  captured  using  digital 
processes  that  capture  images  behind  the  glass  of  a  computer  monitor,  hence  the  “glass  to 
glass”  evolution.  (Chase,  1994) 

This  chapter  consists  of  a  review  of  similar  studies  that  have  been  performed  in  the 
past  and  compares  how  this  research  project  differs  from  previous  studies.  This  chapter 
sets  up  the  follow-on  chapters  that  compare  the  cost  effectiveness  of  a  film-based  system 
to  a  digital  system. 

A.  PRELIMINARY  PAYBACK  ANALYSIS 

In  a  preliminary  payback  analysis,  the  Project  Office  contracted  with  Sherikon  to 
verify  prior  predictions  of  potential  cost  benefits  from  MDIS,  based  on  micro-analyses 
conducted  by  independent  researchers.  In  the  Sherikon  study,  the  costs  were  broken 
down  into  first-order  effects,  second-order  effects,  and  third-order  effects  based  on  a 
Scientific  American  article  by  Malone  and  Rockart  (1991).  The  article  describes  some  of 
the  universal  benefits  seen  by  organizations  which  have  installed  computer  networking 
information  technology  systems  into  the  work  place  to  reduce  costs,  improve  productivity, 
and  create  innovative  ways  of  doing  business.  First-order  effects  are  direct  costs  that  are 
avoided  as  a  result  of  the  information  technology  system.  These  costs  may  include 
material  and  labor.  The  article  describes  the  first-order  effect  as,  “reducing  coordination 
costs  by  the  substitution  of  information  technology  for  human  coordination.”  The  second- 
order  effects  describe  more  intangible  benefits  such  as  productivity  improvements.  Finally, 
third-order  effects  represent  the  long  term  benefits  of  the  system.  An  example  of  a  third- 
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order  effect  is  innovations  that  improve  products  or  processes  that  would  not  be  possible 
without  the  new  information  system. 

The  analysis  calculated  a  system  payback  period  of  5.1  years  after  total 
implementation.  The  net  present  value  for  the  system  was  $9.8  million  and  a  benefit  to 
cost  ratio  was  2.19  (Sherikon,  1994).  Unfortunately,  there  are  some  serious  flaws  with 
the  results.  The  cost  figures  for  film  savings  are  based  on  a  totally  filmless  system. 
However,  Madigan  runs  a  dual  system  at  the  present  time,  generating  both  film  copies  and 
soft-copy  images  so  there  are  no  savings  on  film.  Many  of  the  results  reported  are 
inaccurate  because  of  this  assumption. 

In  Sherikon’ s  payback  analysis,  the  project  office  hoped  to  verify  prior  predictions 
by  independent  researchers  based  on  data  from  the  Digital  Imaging  Network  System 
project.  As  more  sites  receive  the  MDIS  system,  more  accurate  cost  and  benefit  analyses 
can  be  conducted  to  reflect  the  changing  costs  and  benefits  associated  with  MDIS  systems 
in  DoD  medical  treatment  facilities.  In  large  developmental  projects  such  as  MDIS,  the 
cost  and  benefit  data  evolve  as  the  technology  matures.  The  data  collected  during  this 
research  will  be  used  to  validate  other  cost  benefit  studies.  It  represents  a  snap  shot  in 
time  of  the  evolving  digital  imaging  technology. 

This  research  project  grew  out  of  the  Futures  section  of  the  Sherikon  study.  The 
following  is  the  list  of  future  research  areas  in  the  Sherikon  study: 

•  Reading  Times 

•  Interpretation  Times 

•  Retake  Rates 

•  Turnaround  Times 

•  Physician  Acceptance 

•  Image/Report  Availability 

•  Teleradiology 

•  Telemedicine 

•  Silver  Recovery 

•  Handling  and  Storage  Costs 
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•  Paper  Processing 

•  Lost  Images  and  Reports 

•  Water-soluble  Contrast  Media. 

The  italicized  bullets  were  chosen  for  this  project  because  they  are  independent  of 
the  duality  (film  and  digital  images)  that  exists  within  the  Radiology  Department  at 
Madigan. 

B.  ECONOMIC  ANALYSIS  FOR  MDIS  AT  THE  NEW  BROOKE  ARMY 

MEDICAL  CENTER 

An  economic  analysis  was  performed  by  project  office  personnel  to  determine  the 
best  course  of  action  for  the  new  hospital  at  Brooke  Army  Medical  Center,  in  San 
Antonio,  Texas.  The  study  considered  three  configurations  for  the  Radiology  Department 
in  the  new  hospital  facility.  They  are  as  follows: 

•  Alternative  1 :  Incorporates  the  existing  MDIS  equipment  into  the  new  facility 
(a  20%  filmless  facility), 

•  Alternative  2:  Move  existing  equipment  to  the  new  hospital  and  add  enhanced 
equipment  (an  80%  filmless  facility), 

•  Altemative3:  Use  a  conventional  film-based  Radiology  Department. 

To  select  the  best  alternative,  the  study  analyzed  incremental  costs,  that  is  the  cost 
differences  associated  with  each  alternative.  The  study  provides  exhaustive  cost 
information  on  film  costs,  chemical  costs,  storage  costs,  and  associated  equipment. 

The  study  also  reports  on  the  successful  implementation  of  digital  imaging 
technology  to  date.  A  radiologist  at  Brooke  indicates  that  digital  imaging  has  improved 
the  image  loss  rate  from  20%  to  less  than  1%  and  decreased  the  image  retake  rate  from  a 
range  of  8-12%  to  less  than  5%.  The  time  to  retrieve  an  image  from  the  MDIS  system  at 
a  workstation  is  2  to  3  minutes  for  images  archived  on  optical  storage  media.  (Winters, 
1994)  The  installation  at  Brooke  is  successful  in  incorporating  several  modalities, 
including:  computed  radiography,  computed  tomography,  ultrasound,  and  angiography. 
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This  research  hoped  to  obtain  similar  results  when  comparing  Madigan  with  Fitzsimons 
Army  Medical  Centers. 

The  Brooke  Army  Medical  Center  analysis  found  that  alternative  1  provided  a 
$113,325  cost  savings  using  a  20%  filmless  operation.  Alternative  B  cost  $4  million  for 
enhanced  equipment  but  saves  $481,630  by  using  an  85%  filmless  operation.  A  film-based 
system.  Alternative  C,  cost  $.48  million  for  reprinting  digital  images  but  saved  $3.8  million 
in  maintenance,  utilities,  and  personnel.  However,  Alternative  C  incurred  additional  costs 
to  redesign  and  reconstruct  a  Radiology  Department  in  the  new  facility  that  could  process 
and  store  film. 

One  of  the  more  significant  cost  benefits  of  digital  imaging  discussed  in  the  study 
involved  avoided  legal  costs.  The  research  indicated  that,  in  the  last  two  years,  the  San 
Antonio  Assistant  U.S.  Attorney  estimates  that  a  total  of  $1.5  million  was  paid  out  in 
settlement  or  judgment  for  three  Brooke  medical  malpractice  cases  resulting  from  a  lost 
diagnostic  image.”  (Winters,  1994).  Consideration  was  given  to  other  legal  claims 
involving  lost  images  at  Brooke  and  the  conclusion  drawn  was  that  malpractice 
settlements  cost  the  Government  approximately  $1  million  per  year.  Combining  the  costs 
with  the  benefits  favors  Alternative  2,  moving  the  current  MDIS  equipment  from  the  old 
facility  and  adding  enhancements  to  obtain  an  85%  filmless  Radiology  Department. 

C.  EVALUATION  OF  MDIS  BASED  ON  CLINICAL  EXPERIENCE 

The  first  Department  of  Defense  facility  to  receive  the  MDIS  system  was  Madigan 
Army  Medical  Center  in  March  1992.  Madigan  is  installing  the  second  phase  of  a  three 
phase  MDIS  implementation  schedule.  The  two  articles  discussed  previously  represent 
empirical  research  in  the  area  of  cost  benefit  and  cost  effectiveness.  The  remaining  articles 
discussed  in  this  Chapter  report  on  the  performance  of  the  system  under  operational 
conditions. 

An  article  written  by  one  of  the  system’s  early  radiology  users  reports  both  good 
and  bad  lessons  learned  by  the  first  operational  users.  On  the  good  side,  the  system  has 
archived  more  than  375,000  computed  radiography  (CR)  images  within  the  system’s 
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operational  life.  The  MDIS  system  has  improved  the  medical  treatment  facility’s  ability  to 
store  and  retrieve  images.  (Smith,  1994)  Previous  studies  indicated  that  on  the  average  a 
large  treatment  facility  such  as  Madigan  lost  up  to  38%  of  the  images  taken  (Leckie, 
1992). 

In  addition,  fail-safe  strategies  have  been  continually  improved  to  maintain 
continuous  clinical  image  availability  during  times  when  the  MDIS  system  or  components 
malfunction.  After  two  years  of  using  digital  imaging,  Madigan  reports  a  1%  image  loss 
rate,  primarily  due  to  computed  radiography  interface  errors  or  human  errors.  Similarly, 
ergonomic  aspects  of  the  equipment  placement  were  refined  with  time  which  improved 
system  useability.  Furthermore,  unique  patient  cases  can  be  assimilated  into  teaching  files, 
improving  the  physician  training  program.  Finally,  the  MDIS  system  can  correct  an  over 
or  under  exposed  image  through  electronic  window  and  leveling  techniques  (adjustment  of 
the  brightness  and  contrast  on  the  computer  monitor),  providing  for  a  diagnosis  without 
retaking  the  image.  (Smith,  1994) 

On  the  downside,  the  system  has  not  been  successful  in  networking  several  of  the 
other  imaging  modalities,  including  magnetic  resonance,  nuclear  medicine,  ultrasound, 
angiography,  and  the  cardiac  catheterization  lab  due  to  software  interface  problems. 
(Smith,  1994)  Another  operational  problem  with  the  system  involves  the  radiologists’ 
need  for  hardcopy  film.  Throughput  is  the  process  whereby  the  system  preloads  images 
for  the  radiologists’  diagnosis.  At  Madigan,  the  system  cannot  provide  adequate 
throughput  of  images  to  keep  the  radiologists  busy  enough  to  meet  the  demands  of  the 
workload  within  the  department.  However,  Brooke  Army  Medical  Center  also  installed 
the  MDIS  system  and  the  Radiology  Department  has  almost  completely  converted  to 
softcopy  diagnosis.  Their  Radiology  Department  has  obtained  a  20%  filmless  operation 
and  hopes  to  obtain  an  85%  filmless  operation  with  additional  MDIS  equipment  (Winters, 
1994). 
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m.  METHODOLOGY 


A.  DEFINING  THE  PROBLEM 

The  purpose  of  this  research  is  to  analyze  three  measures  of  effectiveness  (MOEs) 
for  the  MDIS  system.  The  three  MOEs  or  measurement  criteria  are  retakes,  physician 
waiting  time  for  archival  image  retrieval,  and  number  of  images  lost  within  a  facility.  This 
analysis  compares  measurable  information  that  is  available  from  Madigan  Army  Medical 
Center  for  digital  imaging  technology  with  a  conventional  radiology  system  at  Fitzsimons 
Army  Medical  Center.  Many  other  criteria  can  be  measured  in  the  future  as  the  system’s 
phased  installation  is  completed. 

B.  PROJECT  SCENARIO 

The  MDIS  system  is  an  information  management  system  which  primarily  supports 
the  physician  in  diagnosing  and  treating  diseases  which  require  medical  imaging 
modalities.  The  system  digitizes  the  images  and  stores  them  on  optical  discs  for  future 
use,  eventually  eliminating  the  need  for  hardcopy  film  and  record  storage  areas.  The 
modalities  currently  available  at  Madigan  include  computed  radiography,  fluoroscopy,  and 
digital  spot  imaging.  The  indefinite  requirements  contract  calls  for  integrating  other 
modalities  as  well,  including:  magnetic  resonance  imaging,  computed  tomography, 
ultrasound,  and  nuclear  medicine.  Software  problems  have  slowed  the  implementation 
process  and  to  date  it  is  unknown  when  the  other  modalities  will  be  integrated  within  the 
system.  Hence,  this  research  effort  only  considers  a  portion  of  the  true  cost  information 
(retakes,  archival  images,  lost  images). 

The  digital  imaging  facility  used  in  this  research  project  is  Madigan  Army  Medical 
Center  located  in  Tacoma,  Washington.  Madigan  opened  a  new  hospital  at  Fort  Lewis  in 
February  1992.  The  original  hospital,  built  in  1944,  was  a  sprawling,  one  story 
cantonement  style  building  designed  to  prevent  the  spread  of  disease  and  fires.  Today  the 
hospital  supports  99,000  active  duty  members,  their  dependents,  and  military  retirees.  The 
new  facility  is  equipped  with  many  innovative  technologies  including:  energy  efficient 
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skylights,  a  medical  mall  (with  “one  stop  shopping  for  your  health  care’),  a  robotic 
medical  material  transport  system,  and  the  medical  digital  imaging  and  support  (MDIS) 
system.  The  hospital’s  average  daily  workload  consists  of  approximately  40  surgeries, 
29,500  laboratory  procedures,  3,700  perscriptions  filled,  2,600  images  taken,  and  7  babies 
delivered.  The  staff  consists  of  1400  civilians,  700  officers  (400  physicians),  and  800 
enlisted.  (Public  Affairs  Office,  1995) 

C.  BASELINE  SCENARIO 

The  baseline  facility  used  in  this  research  is  Fitzsimons  Army  Medical  Center 
located  in  Aurora  Colorado.  The  medical  treatment  facility  has  multiple  missions: 
maintain  or  restore  health,  train  health  care  providers,  provide  quality  health  care,  and 
obtain  excellence  through  research.  The  hospital  serves  57,960  patients  within  a  40  mile 
catchment  area.  The  hospital  provides  tertiary  care  and  consultant  responsibilities  in  a  14- 
state  region.  There  are  155  military  physicians,  22  civilian  physicians,  and  121  continuing 
medical  education  physicians.  The  facility  averages  47,000  outpatient  visits  per  month. 
(Public  Affairs  Office,  1994) 

The  radiology  department  primarily  uses  conventional  radiology  imaging 
equipment  (film-based  technology).  They  also  have  a  mini-PACs  system  which  enables  the 
radiologist  to  store  and  retrieve  images  using  a  computer  system.  The  Chief,  Radiology 
plans  to  have  personal  computers  modified  and  networked  with  the  mini-pacs  so  that 
physicians  will  be  able  to  view  the  images  at  their  worksites. 

D.  JOINT  TACTICAL  COMMUNICATIONS  (TRI-TAC)  METHODOLOGY 

The  Joint  Tactical  Communications  (TRI-TAC)  Office,  located  at  Fort  Monmouth, 
New  Jersey,  is  responsible  for  the  Cost  Effectiveness  Program  which  includes  system 
planning,  trade-offs,  testing,  economic  analyses,  and  cost  analyses  involved  in  the 
acquisition  processes  of  joint  tactical  communications  systems  and  equipment.  They  use  a 
Cost  Effectiveness  Program  Plan  (CEPP)  which  outlines  measures  of  effectiveness,  life 
cycle  cost  estimating,  technology  forecasts,  communications  scenarios,  risk  analyses,  and 
simulation. 
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The  CEPP  is  a  three  volume  document.  The  first  volume  provides  an  overview  of 
the  scope  of  the  plan,  Service  and  agency  relationships,  and  establishes  the  responsibilities 
for  a  joint  Service/ Agency  Cost  Effectiveness  Coordinating  Committee.  The  second 
volume  contains  a  conceptual  model  used  for  this  cost  effectiveness  research  project.  It 
also  provides  guidelines  for  the  modification  and  application  of  the  model  to  a  myriad  of 
new  technology  equipment  alternatives.  The  third  volume  provides  instructions  and 
guidance  for  the  computation  of  life  cycle  cost  estimates.  It  also  provides  formats  for 
reporting  in  compliance  with  Department  of  Defense  financial  regulations  and  policies. 

The  Department  of  Defense  often  uses  the  TRI-TAC  methodology  as  a  decision 
making  tool  in  evaluating  systems.  The  method  yields  a  number  that  tells  the  cost 
effectiveness  per  dollar  of  investment  cost  for  multiple  MOE  assessments.  The  TRI-TAC 
methodology  also  uses  figures  of  merit,  subjective  weighting  criteria,  an  effectiveness 
index,  and  utility  theory. 

System  effectiveness  can  be  expressed  as  one  or  more  figures  of  merit  representing 
the  extent  to  which  the  system  is  able  to  perform  the  intended  function.  The  figures  of 
merit  (FOM)  chosen  for  this  research  project  are  availability,  dependability,  capability,  and 
useability.  Availability  refers  to  the  degree  the  system  is  operable  when  required  by  the 
user  to  perform  a  specific  mission  task  or  procedure.  Availability  is  a  function  of 
operating  time  (reliability)  and  downtime  (maintainability  and  supportability). 
Dependability  is  the  measure  of  the  system  operating  condition  at  one  or  more  points 
when  needed.  Capability  and  useability  are  both  figures  of  merits  which  describe  the 
system’s  performance  parameters.  Examples  of  capability  include  storage  capacity,  image 
clarity,  and  band  of  acceptable  image  quality.  Useability  refers  to  the  ease  with  which  the 
mission  can  be  accomplished  with  each  alternative.  Useability  interfaces  with  the  human 
factor  or  system  operator. 

As  mentioned  previously,  the  three  MOEs  measured  in  this  research  effort  include 
retakes,  time  to  retrieve  historical  images,  and  lost  images.  The  quality  of  the  image 
technology  alternative  is  a  function  of  each  MOE.  The  function  identifying  this 
relationship  is  shown: 
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Q  =  f(R,H,L) 

where, 

R  =  Retakes 
H  =  Historical  Images 
L  =  Lost  Images. 

Each  MOE  is  given  a  utility  value  using  Table  3-1.  The  utility  figure  reflects  the 
relative  performance  of  an  alternative  with  respect  to  the  baseline  alternative.  Therefore, 
Fitzsimons  automatically  gets  a  baseline  score  for  each  MOE.  A  qualitative  assessment  of 
the  digital  system  performance  is  made  for  each  MOE  based  on  the  utility  assignment 
criteria.  The  utility  for  the  alternative,  Madigan,  is  discussed  in  greater  detail  in  the 
analysis  chapter. 


111!! 

— iy 

C8ITEI8A 

0-2 

Barely  meets  minimum  essential  requirements 

2-4 

Less  effective  than  the  baseline 

5 

Baseline 

6-8 

More  effective  than  the  baseline 

9-10 

More  effective  to  the  extent  that  the  MOE  should  be  a  principal 

consideration  in  the  selection  of  a  preferred  alternative. 

Table  3-1.  Utility  Assignment  Criteria 


After  the  utility  values  are  determined  for  each  MOE,  weights  are  assigned  to  each 
figure  of  merit  based  on  its  contribution  to  the  system’s  performance  as  a  whole.  The 
criteria  for  determining  the  relative  contribution  to  the  system  can  be  subjective. 
Therefore,  a  sensitivity  analysis  maybe  necessary.  When  each  FOM  is  weighted  for  each 
MOE,  a  matrix  of  values  is  constructed  to  establish  the  total  FOM  for  each  alternative. 
These  values  are  scored  subjectively  to  determine  the  overall  effectiveness  value. 

The  overall  effectiveness  score  for  each  alternative  is  divided  by  the  equipment  and 
maintenance  costs  for  each  system.  The  operations  costs  are  not  included  in  this  project 
because  Madigan,  the  digital  site,  is  still  running  a  dual  system  which  incorporates  many 
practices  of  the  film-based  technology  system.  A  net  present  value  calculation  for  each 
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alternative  serves  as  the  denominator  when  deriving  the  final  units  of  effectiveness  per 
dollar  investment  for  each  alternative. 

Every  few  years  the  Office  of  Management  and  Budget  (OMB)  publishes  a 
directive  concerning  discount  rates  to  be  used  in  Government  cost  analyses.  For  the 
purposes  of  this  study,  dollar  values  are  given  in  nominal  terms.  Discounted  dollars  help 
determine  the  value  of  future  benefits  and  costs  associated  with  different  alternatives.  The 
most  current  OMB  Circular  Number  A-94  dated  February,  1994  mandates  using  5.5%  for 
spread  sheet  calculations  when  the  time  horizon  for  the  net  present  value  calculation  is 
seven  years. 

E.  CONSTRAINTS  ON  THE  PROJECT 

Budgetary:  The  travel  cost  to  collect  data  for  this  research  is  limited  to  one  trip  to 
each  facility  to  establish  points  of  contacts,  collect  information,  and  take  measurements. 
The  total  cost  of  this  travel  is  approximately  $1800. 

Data  Collection  Sites:  Because  the  MDIS  system  is  new  there  are  only  two 
operational  sites  from  which  to  collect  information:  Brooke  Army  Medical  Center,  and 
Madigan  Army  Medical  Center.  Madigan  was  selected  for  this  study  because  it  has  had 
the  system  the  longest  of  the  two  facilities. 

F.  DATA  COLLECTION 

Data  was  collected  primarily  via  questionnaires  and  interviews  with  hospital 
personnel  at  the  two  Army  Medical  Centers.  Questionnaires  and  interviews  were  chosen 
as  the  collection  instruments  because  the  respondents  are  the  subject  matter  experts 
regarding  imaging  technology.  Therefore,  this  research  focuses  on  the  primary  users  of  the 
MDIS  system,  the  physician.  The  research  assumes  that  the  two  medical  treatment 
facilities  have  similar  workload  data. 

The  dendritic  analysis  technique  is  used  to  decompose  each  measure  of 
effectiveness  into  their  critical  components.  The  dendritic  analysis  approach  (Test  and 
Evaluation  Guide,  1993)  is  used  to  chart  the  primary,  subsidiary,  and  tertiary  research 
questions  and  the  test  measurement  technique  used  to  answer  the  questions  (Figure  3-1). 
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|  Are  there  cost 
effectiveness 
differences  between  / 
digital  imaging 
and  film-based 
imaging  systems? 


Does  digital  imaging  reduce  the  amount  of  retakes 

when  compared  to  the  ^  What  are  the  causes  of  retakes? 

traditional  film  based  system? 

What  percent  of  patient  images  require 
retakes? 


H ^  Interview  (Radiology) 


|  Does  digital  imaging  reduce  the  number  of  images 


lost  within  an  MTF? 


►  What  percent  of  archival  images  are  not 
available  when  requested  by  a  clinician? 

What  are  the  consequences  of  lost  images? 

What  are  the  causes  of  lost  images? 


Questionnaire 

Interview  (Radiology  Departments) 
— ►  Questionnaire 


— ►  Independent  Research 

Army  Surgeon  General’s  Office 
— ►  Questionnaire 


Does  digital  imaging  improve  physician  efficiency 

by  reducing  time  to  retrieve  images?  ^  ^  Of  the  personnel  listed  below,  place  a  number  from 

1  to  5  in  the  boxes  to  indicate  the  person  who 
most  frequently  retrieves  images  from  the  image 
storage  area?  - ►  Questionnaire 


On  the  average,  how  much  time  does  it  take  to 
retrieve  an  image  from  your  organizations  record 
storage  location?  - 


What  are  the  annual  costs  for  both  systems? 


What  is  the  total  outpatient  load  at  each 
facility? 

^  What  are  the  investment  costs? 

What  are  the  maintenance  costs? 

What  are  the  utility  scores  for  each 
alternative? 

What  figure  of  merits  will  be  used  to 
measure  system  effectiveness? 

What  weights  will  be  assigned  to  each 
figure  of  merit? 


Questionnaire 
Biostatistidan  Interview 

MDIS  Project  Office 
Biomedical  Maintenance 


-►  Dedsion  Maker  Evaluation 


Dedsion  Maker  Evaluation 
Dedsion  Maker  Evaluation 


Figure  3-1.  Dendritic  Approach  to  Cost  Effectiveness  Analysis 


G.  QUESTIONNAIRE 


The  questionnaire  was  validated  through  a  pilot  test.  The  questionnaire  was  used 
to  answer  the  questions  in  the  Dendritic  Approach  (Figure  3-1).  The  questionnaire  was 
first  administered  to  clinicians  in  Army  Medical  Centers  other  than  Madigan  and 
Fitzsimons.  The  pilot  test  validated  the  numbers  and  thresholds  in  the  survey’s  multiple 
choice  questions.  Three  Army  Medical  Centers  participated  in  the  pilot  test  including 
Brooke,  Tripler,  and  Eisenhower  and  the  clinical  respondents  provided  little  constructive 
criticism. 
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The  questionnaire  was  administered  to  physicians  who  use  the  Radiology 
Department  at  each  site.  Physician  opinions  concerning  their  experience  with  the  imaging 
technology  present  at  their  facility  was  sought.  The  desired  sample  size  was  a  minimum  of 
30  respondents  at  each  medical  facility.  The  Central  Limit  Theorem  provides  a  rule  of 
thumb  stating  when  the  sample  size  (n)  is  >  30;  the  sample  has  approximately  a  normal 
distribution.  In  an  analysis  of  variance  test,  one  of  the  assumptions  made  is  that  the 
population  or  treatment  distributions  are  all  normal  with  the  same  variance.  By  obtaining 
a  sample  size  greater  than  30,  normality  can  be  assumed  by  the  Central  Limit  Theorem. 
(Devore,  1991) 

The  technique  used  to  select  respondents  to  the  questionnaire  is  a  judgmental 
sample.  Questionnaires  were  given  to  physicians  who  volunteered  their  time  (see 
Appendix  A,  sample  questionnaire).  The  questionnaire  was  administered  to  physicians 
who  worked  with  the  Department  of  Radiology  at  the  two  medical  treatment  facilities  in 
October,  1994.  Approximately  100  questionnaires  were  distributed  at  each  site.  At 
Madigan,  the  response  rate  was  58  responses  out  of  a  pool  of  400  (14.5  %).  At 
Fitzsimons,  the  response  rate  was  36  responses  out  of  295  (12.2%).  The  analysis  assumes 
that  the  opinions  of  the  respondents  are  representative  of  the  population  at  each  site.  The 
consolidated  questionnaire  data  collected  are  shown  in  Appendix  B. 

H.  INTERVIEWS 

Interviews  with  hospital  personnel  were  conducted  at  the  work  site  after 
completing  the  questionnaires.  The  interviews  collected  specific  information  for  the 
research  project  that  was  not  available  from  the  questionnaire  responses.  The  information 
validated  and  extended  the  information  in  the  questionnaires.  Interviews  were  primarily 
conducted  with  hospital  support  staff  and  remained  anonymous  if  requested;  otherwise  the 
information  used  in  the  study  is  quoted  from  the  source.  Follow-on  telephone  interviews 
were  conducted  as  needed  using  the  statement  in  Appendix  C. 
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IV.  ANALYSIS 


The  analytical  portion  of  this  research  demonstrates  both  quantitatively  and 
qualitatively  whether  or  not  the  MDIS  system  is  cost  effective  for  the  Army  based  on  data 
collected  on  three  measures  of  effectiveness  (MOEs).  The  three  MOEs  are:  retakes, 
physician  waiting  time  for  archival  image  retrieval;  and  number  of  images  lost  within  a 
facility.  A  comparison  between  two  similar  medical  treatment  facilities  focusing  on 
physician  opinions  will  be  the  basis  for  this  analysis.  First,  the  three  MOEs  will  be 
analyzed  to  identify  the  trends  and  the  associated  costs  for  each.  Questionnaire  data  are 
used  in  the  following  paragraphs  to  answer  the  subsidiary  research  questions.  Factual 
interview  data  are  used  in  spreadsheet  computations.  Next,  a  modified  life  cycle  cost  will 
be  computed  for  each  alternative  using  maintenance  and  capital  investment  costs  only. 
Finally,  units  of  effectiveness  per  dollar  investment  will  be  determined  for  each  alternative 
using  the  TRI-TAC  Methodology  discussed  in  Chapter  III. 

A.  DOES  DIGITAL  IMAGING  REDUCE  THE  AMOUNT  OF  RETAKES 

WHEN  COMPARED  TO  THE  TRADITIONAL  FILM-BASED  SYSTEM? 

Image  retakes  occur  for  a  variety  of  reasons.  The  major  categories  for  image 
retakes  within  the  Radiology  Department  at  Fitzsimons  include  over  or  under  exposure, 
fog  film,  poor  patient  positioning,  processing  error,  artifacts  on  film,  machine  error,  and 
technician  error  (see  Appendix  B).  At  Fitzsimons,  poor  patient  positioning  causes  about 
50%  of  the  retakes.  The  second  largest  category  of  retakes  is  over  and  under  exposure  of 
the  radiographic  plates.  This  accounts  for  25%  of  retakes  at  Fitzsimons  (Leonard,  1994). 
Each  of  these  causes  can  be  improved  with  the  implementation  of  digital  imaging 
technology.  Under  and  over  exposed  film  can  be  accomodated  with  window  and  leveling 
techniques  and  a  diagnosis  can  be  obtained.  As  clinicians  become  more  proficient  with 
this  technology,  poor  patient  positioning  can  be  overcome  by  manipulating  the  digital 
image.  Both  poor  positioning  and  under/over  exposure  of  film  should  result  in  a 
significant  difference  between  digital  imaging  and  film-based  imaging  technologies. 
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Retake  rates  at  Madigan  are  captured  in  a  much  less  rigid  manner  because  some 
sections  are  filmless  and  use  only  softcopy  images  while  the  majority  of  sections  still 
produce  hard  copy  film.  Softcopy  retake  data  are  not  captured  and  are  not  reflected  in  the 
figures  provided  by  the  Madigan  Radiology  staff.  (Carter,  1994)  With  the  digital  imaging 
technology,  films  can  be  processed  over  a  wider  range  of  exposure  times.  This  results  in  a 
more  forgiving  system  than  conventional  radiology  in  terms  of  over  and  under  exposed 
films.  With  conventional  radiography,  a  natural  feedback  mechanism  exists  in  which  an 
underexposed  film  is  too  light  whereas  an  overexposed  film  is  too  dark. (Smith,  1994) 

Retake  data  were  collected  at  the  two  medical  treatment  facilities  to  determine 
whether  the  digital  imaging  technology  significantly  changed  the  number  of  retakes.  Both 
facilities  were  able  to  provide  10  months  of  retake  figures  expressed  in  percentage  of 
images  taken  in  the  calendar  year,  1993.  The  graph  shown  below  (Figure  4-1)  depicts  the 
month  by  month  comparison  between  the  two  MTFs.  The  zeros  depict  months  in  which 
the  data  were  not  available.  Madigan  did  not  have  figures  for  October  and  November  due 
to  system  downtime  and  Fitzsimons  did  not  have  the  retake  figures  for  January  and 
February. 

An  interesting  trend  appeared  in  the  data  provided  by  Madigan.  The  digital  system 
became  operational  in  the  early  part  of  1992  and  retake  figures  were  captured  for  the  last 
6  months  in  1992  with  a  mean  retake  rate  of  7.9%.  In  1993,  the  retake  rate  began  at  5.9% 
and  steadily  declined  to  2%.  This  suggests  a  learning  curve  which  over  time  will  show 
significant  differences  between  digital  imaging  technology  and  conventional  imaging 
technology. 
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Retake  Analysis  1993 


JAN  FEB  MAR  APR  MAY  JUN  JUL  AUG  SEP  OCT  NOV  DEC 

Month 

Figure  4-1.  Retake  Analysis  for  Madigan  and  Fitzsimons  1993 
A  one-way  analysis  of  variance  (ANOVA)  with  a  95%  confidence  interval  was 
conducted  to  determine  if  the  retake  rates  at  the  two  different  facilities  were  statistically 
different.  The  null  hypothesis  states  that  there  is  no  difference  between  the  Madigan 
retake  rate  and  the  Fitzsimons  retake  rate  as  a  result  of  the  different  imaging  technologies. 
The  p  value  is  much  greater  than  the  .05  value  necessary  to  reject  the  null  hypothesis. 
Therefore,  based  on  the  retake  rates  provided  for  1993,  there  is  no  significant  difference  in 
the  amount  of  retakes  generated  by  the  different  imaging  technologies  (see  Figure  4-2). 
This  finding  is  further  substantiated  by  a  questionnaire  administered  to  the  physician  staff 
at  both  facilities.  The  majority,  65%  of  the  MAMC  staff  and  61%  of  the  FAMC  staff  felt 
that  the  retake  rate  was  between  1-10%  (see  Appendix  B). 
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Anova:  Single  Factor 

SUMMARY 

Groups 

Count 

Sum 

Average 

Variance 

MAMC 

10 

45.2 

4.52 

2.132889 

FAMC 

10 

39 

3.9 

0.773333 

ANOVA 

Source  of  Variation 

SS 

df 

MS 

F 

P-value 

Fcrit 

Between  Groups 

1.922 

1 

1.922 

1.322679 

0.265163 

4.413863 

Within  Groups 

26.156 

18 

1.453111 

Total 

28.078 

19 

Figure  4-2.  ANOVA  Comparing  Madigan  and  Fitzsimon  Retake  Rates  1993 
The  data  that  were  collected  are  not  inconsistent  with  the  information  stated,  if 
consideration  is  given  to  the  possibility  of  learning  to  use  the  system  to  its  full  capability, 
particularly  in  terms  of  under/over  exposure  and  poor  patient  positioning.  Figure  4-1  only 
considers  a  strict  average  of  all  monthly  retake  data.  If  one  considers  the  learning  process 
then  the  early  months  skew  the  true  average  and  the  true  average  is  more  realistically 
lower  when  one  averages  the  last  six  months  of  data  collected  in  1993.  Validation  of  this 
theory  could  best  be  accomplished  by  collecting  the  retake  rates  at  both  facilities  for  1994. 
However,  this  information  was  not  available  for  this  project. 

In  summary,  the  results  of  the  questionnaire  do  not  show  a  significant  difference 
between  MDIS  and  film-based  technology  with  regard  to  retake  figures  based  on  a  simple 
average.  The  data  clearly  show  a  trend  of  decreasing  retake  rates  at  Madigan,  possibly 
due  to  the  learning  associated  with  window  and  leveling  techniques  and  making  a 
diagnosis  from  an  image  that  was  not  positioned  correctly.  More  data  need  to  be 
collected  to  validate  this  hypothesis. 

B.  DOES  DIGITAL  IMAGING  REDUCE  THE  NUMBER  OF  IMAGES  LOST 
WITHIN  AN  MTF? 

Under  a  conventional  radiology  system,  there  are  many  reasons  that  records  are 
lost.  For  example,  both  Madigan  and  Fitzsimons  are  Department  of  the  Army  teaching 
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institutions  in  which  many  resident  physicians  are  assigned.  Consequently,  images  are 
frequently  retained  by  physicians  to  provide  future  teaching  files.  Many  patients  do  not 
trust  the  filing  system  and  check-out  images  without  returning  them  to  the  medical 
treatment  facility.  The  frequency  of  lost  images  is  also  a  function  of  the  size,  capacity,  and 
population  supported  by  a  facility.  Often  records  end  up  in  clinics,  physician  offices,  ward 
administrative  areas,  or  within  “black  holes.”  The  final  major  cause  of  lost  images  is 
administrative  type  errors  such  as  mixing  up  record  jackets,  misfiling  images,  or  recording 
incorrect  patient  data  on  an  image. 

A  study  conducted  in  1992  by  Dr.  Bob  Leckie,  radiologist,  indicated  that  before 
the  MDIS  system  was  installed  at  Madigan  up  to  38%  of  patient  images  were  unavailable 
for  the  physician  when  requested.  Follow-on  studies  at  other  major  medical  centers 
(Tripler,  and  Brooke)  revealed  similar  loss  rates  (Leckie,  1992).  The  results  of  the 
questionnaire  responses  from  physicians  at  Fitzsimons  support  Leckie’ s  data.  The 
majority  of  physicians  felt  that  images  were  unavailable  to  them  between  5-20%  of  the 
times  requested.  With  the  digital  imaging  system,  the  majority  of  the  physicians  (29%)  at 
Madigan  felt  that  archived  images  were  unavailable  due  to  loss  less  than  5%  of  the  times 
requested.  On  the  other  hand,  many  physicians  at  Fitzsimons  (33%)  felt  that  images  were 
unavailable  due  to  loss  5-10%  of  the  times  requested.  (Appendix  B) 

Although  difficult  to  assess  a  cost  saving,  the  most  severe  consequence  of  a  lost 
image  is  delayed  diagnosis  and  treatment,  which  can  result  in  death.  Some  other 
unfortunate  consequences  include  inappropriate  treatment,  poor  quality  patient  care,  and 
increased  radiation  exposure  to  patients  where  images  are  retaken.  Legal  claims  for 
military  dependents  and  retirees  may  result  from  any  of  the  above  consequences.  The 
magnitude  of  the  legal  costs  associated  with  those  claims  was  demonstrated  at  Brooke 
Army  Medical  Center  in  the  Winters  Economic  Analysis  discussed  in  Chapter  II  (Winters, 
1994). 

The  results  of  the  questionnaire  validate  previous  studies  indicating  that  digital 
imaging  technology  decreases  the  number  of  images  lost  within  a  facility.  The  majority  of 
physicians  at  Madigan  felt  that  images  were  unavailable  less  than  5%  of  the  times 
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requested,  a  significant  improvement  over  Leckie’s  findings  (38%  loss  rate).  Specific 
benefits  are  difficult  to  determine  due  to  the  qualitative  nature  of  the  consequences  of  lost 
images. 

C.  DOES  DIGITAL  IMAGING  IMPROVE  PHYSICIAN  EFFICIENCY  BY 

REDUCING  TIME  TO  RETRIEVE  IMAGES? 

A  physician’s  time  is  very  valuable.  Finding  ways  to  improve  physician  efficiency 
will  generate  qualitative  as  well  as  quantitative  savings  in  military  medical  treatment 
facilities.  The  frequency  of  response  to  question  2,  “On  the  average  how  much  time  does 
it  take  to  retrieve  an  image  from  your  storage  location?”  shows  a  significant  difference 
between  facilities.  The  largest  percentage  of  physicians  at  Madigan  (47%)  reported  that 
archival  images  are  available  to  them  within  10  minutes.  By  comparison,  42%  of  the 
physicians  at  Fitzsimons  responded  that  it  takes  more  than  2  hours  to  retreive  an  image 
from  the  Radiology  records  storage  area.  (Appendix  B) 

An  analysis  of  the  clinic  outpatient  visits  at  Madigan  shows  an  additional  26,000 
outpatient  visits  between  1991,  when  Madigan  used  conventional  film  technology,  and 
1993,  when  Madigan  used  digital  technology  (see  Figure  4-3).  It  is  unclear  what  caused 
this  additional  outpatient  workload.  The  number  of  assigned  physicians  has  not  increased 
but  the  in-house  staff  is  treating  more  patients  on  an  outpatient  basis.  The  improved 
physician  efficiency  may  be  due  in  part  to  the  ability  to  retrieve  archived  images  faster. 
Conversely,  outpatient  visits  at  Fitzsimons  have  decreased  dramatically  over  the  same  time 
period.  The  cause  of  this  decrease  is  also  unclear,  but  it  could  be  related  to  the  decreasing 
patient  population  at  Fitzsimons  Army  Medical  Center  due  to  downsizing  at  nearby 
military  bases.  Recently  the  hospital  was  included  on  the  Base  Realignment  and  Closure 
list.  This  thesis  assumes  that  using  conventional  technology  has  not  caused  the  decrease  in 
outpatient  visits  at  Fitzsimons. 

Detailed  data  were  unavailable  to  draw  specific  conclusions  about  physician 
efficiency  as  related  to  the  digital  imaging  system.  In  the  future,  the  teleradiology 
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initiative,  “Project  Seahawk”,  hopes  to  demonstrate  improved  physician  efficiency  through 
the  use  of  remote  consultations  and  radiologic  diagnoses. 


Figure  4-3.  Number  of  Outpatient  Visits 

The  results  of  the  questionnaire  clearly  show  that  physicians  can  retrieve  archived 
images  faster  under  a  digital  technology  imaging  system.  It  remains  unclear  as  to  whether 
this  time  saving  tool  improves  physician  efficiency  because  data  are  not  available  to 
validate  how  physicians  at  Madigan  use  their  time. 

D.  MODIFIED  LIFE-CYCLE  COST  CALCULATION 

To  calculate  cost  effectiveness,  one  must  begin  by  examining  the  overall  system 
cost.  For  the  purposes  of  this  research,  the  system  cost  only  includes  the  investment 
equipment  cost  and  maintenance  cost.  Direct  costs  such  as  film,  processors,  and  film 
processing  chemicals  are  not  included  in  the  system  cost  because  Madigan  is  running  a 
redundant  system  using  both  digital  imaging  and  a  film  back-up.  Operations  costs  are  also 
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not  considered  due  to  the  uncertainty  associated  with  staffing  when  Madigan  goes  to  a 
strictly  filmless  radiology  department.  To  date,  it  is  unknown  whether  digital  imaging 
technology  will  reduce  staffing  requirements.  Therefore  operations  costs  are  excluded 
from  the  analysis. 

The  MDIS  system  is  a  joint  Service  acquisition  project  with  the  Army  Medical 
Department  currently  acting  as  the  lead  agent.  An  indefinite  requirements  contract  was 
selected  for  this  procurement  because  the  project  office  wanted  to  assess  the  commercial 
product  before  deciding  to  place  more  orders  against  the  contract.  With  this  fixed-price 
type  of  contract,  the  Government  can  place  orders  without  discussing  price,  quantity,  or 
performance  period  with  the  contractor  (Contingency  Contracting,  1993).  If  the 
Government  is  not  satisfied  with  the  product  performance,  there  is  no  obligation  to  place 
any  more  delivery  orders.  In  this  way,  the  project  office  can  control  how  many  systems 
will  be  fielded  to  DoD  medical  treatment  facilities. 

To  date,  the  project  office  has  issued  11  delivery  orders  against  the  indefinite 
delivery  order  contract.  The  contract  was  signed  in  September  1991  and  was  supposed  to 
expire  in  September  1994;  however,  the  contract  was  recently  extended  for  one  year. 
Progress  payments  are  made  to  the  contractor  based  on  a  milestone  concept.  There  are 
three  phases  for  the  total  installation  and  each  phase  has  four  major  steps,  including 
receipt  of  equipment,  installation  of  equipment,  system  integration,  and  total  integration. 
Each  of  these  major  steps  has  specific  milestones.  When  a  milestone  is  80%  complete,  an 
80%  payment  is  made  to  the  contractor.  The  remaining  20%  is  paid  after  each  milestone 
is  completed  and  the  Government  conducts  Acceptance  Testing.  (Romlein,  1994) 

The  total  investment  costs  including  installation,  integration,  and  acceptance 
testing  by  phases  at  MAMC  are  shown  below: 


— 

m 

ail 

Phase  I 

$5,415,923 

Phase  II 

$2,092,999 

Phase  III 

$2,354,556 

Total 

$9,863,478 

Table  4-1.  Total  Investment  Costs  at  MAMC  (Winters,  1995) 
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The  MDIS  contract  contains  a  one  year  warranty  clause  from  the  day  of 
Government  acceptance.  None  of  the  phases  at  the  MAMC  site  have  been  completed. 
Therefore,  the  warranty  period  has  not  started  on  any  phase  of  the  MAMC 
implementation  process.  The  maintenance  costs  begin  accruing  when  the  warranty  period 
expires  and  will  be  phased  in  upon  completing  Government  acceptance  testing.  The 
maintenance  costs  negotiated  previously  are  shown  below  in  Table  4-2.  Maintenance 
costs  have  been  separated  by  phases,  spokes  (satellite  medical  treatment  facilities),  and 
specific  equipment  lines  (optimized  workstations  and  RIS  terminals). 


Phase  I  subsystem 

$  363,309 

Phase  II  subsystem 

$  155,463 

Phase  HI  subsystem 

$  110,147 

Fairchild 

$  44,960 

McChord 

$  26,314 

RIS  Terminals 

$  4,356 

Optimized  workstation 

$  1,629 

Total  Maintenance  Cost 

$  706,178 

Table  4-2.  Estimated  Annual  Maintenance  Costs 
for  MAMC  MDIS  System  (Winters,  1995) 


The  cost  of  the  film-based  imaging  system  at  Fitzsimons  is  based  on  the  property 
records  for  each  piece  of  equipment.  The  Radiology  hand  receipt  is  separated  into  eight 
separate  documents  with  a  corresponding  dollar  value  of  $14,796,653.  The  system  used 
to  account  for  the  equipment  does  automatic  depreciation  calculations  in  accordance  with 
applicable  regulations.(0’Bailey,  1995)  The  current  dollar  value  is  not  used  in  the 
modified  life-cycle  cost  spreadsheet  because  the  $14  million  represents  a  sunk  cost.  To 
maintain  the  current  system,  costs  for  capital  investment  equipment,  also  called  extended 
Medical  Care  Support  Equipment  (MEDCASE)  requirements  average  $4.0  million. 
(Mericie,  1995).  The  MEDCASE  manager  provided  the  1993  requirements  as  shown  in 
Table  4-3.  The  table’s  figures  represents  a  high  end  year,  with  costs  close  to  $5  million. 
In  the  two  subsequent  years,  the  MEDCASE  funding  has  dropped  significantly  because 
Fitzsimons  is  a  candidate  for  Base  Closure.  This  thesis  does  not  use  information  from  this 
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period  and  assumes  that  MEDCASE  requirements  would  remain  the  same  if  not 
threatened  by  base  closure. 


ft 

Computerized  Dosimetry  System 

47,369 

Modular  Building 

Upgrade  Laser  Camera  (CT) 

Transcranial  Doppler  System 

19,071 

X-Ray  System  Mobile 

41,500 

Linear  Accelerator 

1,536,850 

X-Ray  Apparatus 

130,000 

Scintillation  Camera 

501,578 

CT 

1,300,000 

Miscellaneous  Support  Equipment 

1,371,452 

File  Storage  System 

125,000 

Total 

$5,258,327.00 

Table  4-3.  Fitzsimons  New  Equipment  Costs  for  1993 


Maintenance  of  the  Radiology  film-based  equipment  is  accomplished  by  the 
biomedical  repair  section.  The  only  complete  year  of  maintenance  costs  the  facility  could 
provide  showed  an  annual  cost  of  $469,768  (McCune,  1995).  The  Chief  of  Biomedical 
Maintenance,  Fitzsimons  breaks  out  the  maintenance  costs  for  1994  as  follows: 


COST 

Man-hour  Costs 

30,312 

Repair  Parts 

34,246 

Contract  Costs 

405,210 

Total 

$469,768 

Table  4-4.  Fitzsimons  Annual  Maintenance  Costs 

The  modified  life-cycle  cost  spreadsheets  are  shown  in  Table  4-5,  for  each  facility. 
For  the  purposes  of  this  study,  only  the  investment  costs  and  maintenance  costs  are 
included  in  the  calculations.  The  discount  rate  used  is  5.5%  in  real  dollars  in  accordance 
with  Office  of  Management  and  Budget  Circular  Number  A-94  (February,  1994). 
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MAMC  Life  Cycle  Cost  ( minus  Operations) 
Total  Investment  Expense  9,863,478 


Maintenance  Cost 

Year  1 

Year  2 

Year  3 

Year  4 

Year  5 

Year  6 

Year  7 

Discounted  $ 

Phase  1  subsystem 

363,309 

363,309 

326,978 

326,978 

290,647 

290,647 

290,647 

1,846,162 

Phase  II  subsystem 

155,463 

155,463 

139,916 

139,916 

124,370 

124,370 

124,370 

789,986 

Phase  III  subsystem 

110,147 

110,147 

99,132 

99,132 

99,132 

88,117 

88,117 

568,140 

Fairchild 

44,960 

44,960 

40,873 

40,873 

36,786 

36,786 

36,786 

230,925 

McChord 

26,314 

26,314 

23,922 

23,922 

21,529 

21,529 

21,529 

135,153 

RIS  Terminals 

4,356 

4,356 

4,356 

4,356 

4,356 

4,356 

4,356 

24,755 

Optimized  workstation 

1,629 

1,629 

1,629 

1,629 

1,629 

1,629 

1,629 

9,258 

Total  Maintenance  Cost 

706,178 

706,178 

636,806 

636,806 

578,449 

567,434 

567,434 

3,604,379 

FAMC  Life  Cvcle  Cost  (minus  Operations) 

Net  Present  Value  MAMC  L 

13.467.857 

Year  1 

Year  2 

Year  3 

Year  4 

Year5 

Year  6 

Year  7 

New  Equipment  Cost 

4,000,000 

3,791,469 

3,593,810 

3,406,455 

3,228,867 

3,060,537 

2,900,983 

2,749,747 

18,665,981 

Maintenance 

Man-hours 

30,312 

28,732 

27,234 

25,814 

24,468 

23,193 

21,984 

20,838 

141,451 

Parts 

40,688 

38,567 

36,556 

34,650 

32,844 

31,132 

29,509 

27,970 

189,870 

Repair  Contracts 

405,210 

384,085 

364,062 

345,082 

327,092 

310,040 

293,877 

278,556 

1,890,911 

Total 

476,210 

451,384 

427,852 

405,547 

384,405 

364,365 

345,369 

327,364 

2,222,232 

Net  Present  Value  FAMC  L 

23.110,445 

Table  4-5.  Modified  Life  Cycle  Cost 

The  net  present  value  for  Madigan  (MAMC)  is  significantly  lower  because  the 
digital  imaging  system  is  new.  The  equipment  replacement  figures  reflect  that  several 
items  will  be  covered  under  a  warranty  for  one  year  within  the  time  horizon  of  the 
spreadsheet  (Romlein,  1995).  The  total  modified  life-cycle  cost  figure  for  each  system  will 
be  used  as  the  denominator  in  the  final  cost  effectiveness  calculation. 


E.  COST  EFFECTIVENESS  CALCULATION 

The  first  step  in  the  TRI-TAC  methodology  for  determining  cost  effectiveness  is  to 
specify  the  utility  values  for  each  measure  of  effectiveness.  The  baseline  figures  will 
simply  be  5  for  each  MOE  as  shown  in  Table  3-1.  The  more  difficult  task  is  to  assign 
values  to  the  alternative,  the  digital  imaging  system.  Retakes  showed  only  a  slight 
difference  when  comparing  actual  rates  reported  in  the  questionnaire  by  the  radiology 
department.  Hence,  the  utility  assignment  for  this  MOE  will  be  a  7.  This  corresponds  to 
the  lower  end  middle  range  score  on  the  utility  assignment  criteria.  The  second  MOE, 
number  of  lost  images,  was  a  significant  attribute  in  both  the  questionnaire  responses  and 
previous  studies  on  lost  images  with  film-based  technology  at  Madigan  (Leckie,  1992). 
The  utility  value  assigned  to  lost  images  is  a  9.  The  third  MOE,  time  to  retrieve  archival 
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images,  showed  a  significant  difference  in  both  the  questionnaire  and  in  the  increased 
number  of  outpatient  visits.  Therefore  it  also  has  a  utility  assignment  of  9. 

The  next  step  is  to  weight  each  of  the  figures  of  merit  for  each  measure  of 
effectiveness.  The  first  MOE  is  retakes;  a  weight  of  .2  is  assigned  for  each  figure  of  merit 
because  they  all  have  approximately  equal  importance  to  the  overall  system.  The  second 
MOE,  lost  images,  has  availability  weighted  most  heavily  with  .3,  dependability,  capability, 
and  useability  weighted  equally  (.2)  and  maintainability  the  lowest  weight  (.1).  The  third 
MOE  is  time  to  retrieve  historical  images.  Availability,  dependability,  and  capability  are  all 
weighted  equally,  at  .3  ,  and  a  lower  weight  is  given  to  maintainability  and  useability  (.05) 
because  these  have  little  impact  on  this  measure.  A  summary  of  the  utility  scores  and 
weights  are  shown  in  Table  4-6  below: 


ft 

"  iiiiiiii 

Ililliil 

MOE 

Retakes  (R) 

5 

7 

Lost  Images  (L) 

5 

9 

Historical  (H) 

5 

9 

Weights  il 

IfvR 

L 

H 

Availability 

0.2 

0.3 

0.3 

Dependability 

0.2 

0.2 

0.3 

Capability 

0.2 

0.2 

0.3 

Maintainability 

0.2 

0.1 

0.05 

Useability 

0.2 

0.2 

0.05 

Table  4-6.  Assignment  of  Utility  and  Weights 
Next  a  total  score  for  each  figure  of  merit  is  calculated  using  the  formula: 


FOMi  = 


£Wj  x  Uj, 

2>, 


where, 

FOMj:  FOM  for  the  ith  alternative 
Wj:  weight  for  the  jth  MOE 

Uji:  utility  assigned  to  the  ^alternative  with  respect  to  the  j* 
MOE 
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Table  4-7  summarizes  each  FOM  total  for  each  alternative.  The  results  are  scored 
according  to  their  relative  importance  and  the  final  overall  effectiveness  score  (E)  is  shown 
for  each  alternative.  The  final  number  represents  the  overall  effectiveness  divided  by  the 


life-cycle  cost  for  each  alternative. 


*4  Weights 

itiiiii 

III  L  :. 

H 

lislilllll 

Bm 

mmmm 

IliiM! 

■■i 

Availability 

0.2 

0.3 

0.3 

5.00 

8.50 

25.00 

500.00 

741 .67 

Dependability 

0.2 

0.2 

0.3 

5.00 

8.43 

30.00 

Capability 

0.2 

0.2 

0.3 

5.00 

5.21 

25.00 

BHimn 

Maintainability 

0.2 

0.1 

0.05 

5.00 

7.86 

5.00 

21.64 

54.94 

Useability 

0.2 

0.2 

0.05 

5.00 

7.11 

15.00 

Total 

37.11 

100.00 

Table  4-7.  Cost  Effectiveness  Calculation 
The  results  of  the  cost  effectiveness  calculation  favor  the  digital  imaging  system  by 
a  factor  of  approximately  2.5  to  1 .  In  terms  of  effectiveness  per  dollar  invested,  the  digital 
system  achieves  over  twice  the  benefit  of  the  conventional  film-based  system.  The  primary 
reason  for  this  difference  is  that  the  life  cycle  cost  is  half  as  much  for  a  digital  system  when 
only  maintenance  and  investment  costs  are  considered.  This  ratio  is  a  conservative 
estimate  because  salvage  value  of  the  system  at  the  end  of  the  time  horizon  was  not 
considered.  If  salvage  value  was  included  it  would  increase  the  cost  difference  between 
the  two  systems,  favoring  the  digital  system.  As  the  operational  proficiency  with  digital 
imaging  improves  at  Madigan,  and  reliance  on  a  film  back-up  system  dissipates,  the 
operations  costs  will  decrease  significantly. 
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V.  CONCLUSION 


A.  GENERAL 

The  MDIS  system  is  a  technological  leap  into  the  future.  The  MDIS  system,  when 
complete,  will  meet  the  goal  of  the  Government’s  healthcare  reform  by  improving 
healthcare  quality  and  providing  a  more  cost  effective  imaging  system.  The  decreasing 
Federal  budget  and  increased  demand  for  quality  warrants  using  cost  effective  state-of- 
the-art  technology  in  the  Department  of  Defense.  The  purpose  of  this  study  was  to 
validate  claims  of  improvements  and  extend  previous  studies  of  the  system.  The  results 
are  based  on  three  measures  of  effectiveness:  image  retakes,  lost  images,  and  time  to 
retrieve  archived  images.  The  two  methods  used  to  collect  the  data  were  questionnaires, 
completed  by  physicians  who  work  under  two  different  imaging  technologies,  and 
interviews  with  specialists  in  logistics,  financial  management,  patient  administration,  and 
information  management  at  each  facility. 

The  sites  chosen  for  the  study  included  Madigan  Army  Medical  Center  and 
Fitzsimons  Army  Medical  Center.  Madigan  is  the  first  medical  treatment  facility  within 
the  Department  of  Defense  to  receive  the  MDIS  system.  It  was  compared  with  the 
baseline  alternative,  Fitzsimons,  which  represents  a  typical  film-based  imaging  system. 
Although  each  facility  supports  a  different  population,  they  are  similar  in  terms  of 
regional  area  mission  responsibilities. 

The  questionnaire  results  were  consolidated  and  recorded  as  frequency  data. 
Interview  data  were  used  in  the  spreadsheets,  tables,  and  graphs.  A  modified  life  cycle 
cost  was  developed  for  each  system  using  capital  investment  and  maintenance  costs.  A 
utility  function  was  developed  for  each  MOE  based  on  quantitative  and  qualitative 
evaluation  of  the  data.  Finally  an  effectiveness  score  was  calculated  for  each  imaging 
system. 
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B.  SUMMARY  OF  RESULTS 


The  overall  questionnaire  results  showed  that  physicians  completing  the 
questionnaire  slightly  favored  digital  imaging  technology.  This  limited  difference  was  not 
surprising,  however.  Madigan  was  the  first  operational  system  user.  The  throughput  rate 
for  retrieving  images  is  not  yet  fast  enough  for  the  radiologist  to  meet  the  proper  diagnosis 
rate.  Under  a  film  based  system,  a  radiologist  can  make  a  diagnosis  every  90  to  180 
seconds  (Bender,  1994).  As  a  result,  Madigan  uses  digital  imaging  with  a  film-based 
back-up  system.  It  runs  a  dual  technology  radiology  imaging  system.  The  system  users’ 
expectations  were  too  high  as  a  result  of  promised  capability  that  has  yet  to  materialize. 
For  example,  the  system  is  not  yet  integrated  with  the  magnetic  resonance  imaging  system. 
Although  digital  images  are  obtained,  they  are  not  yet  networked  within  the  MDIS  system. 
Many  software  upgrades  have  been  installed  causing  some  system  downtime  and 
inconvenience  to  the  users.  The  implementation  schedule  has  slipped  significantly  due  to 
system  integration,  software,  and  requirement  changes. 

C.  RESEARCH  QUESTIONS 

The  primary  research  question  is:  Are  there  cost  effectiveness  differences  between 
the  state-of-the-art  digital  imaging  system  and  the  traditional  film-based  imaging 
technology  most  frequently  used  by  Government  and  civilian  medical  treatment  facilities? 
In  the  final  cost  effectiveness  calculation,  the  digital  imaging  technology  provided  greater 
benefits  than  film-based  imaging  technology  by  a  ratio  of  2.54  to  1.  This  means  that  the 
benefits  per  dollar  investment  for  the  MOEs  used  in  this  study  are  roughly  two  and  a  half 
times  greater  for  digital  imaging  than  film.  The  difference  is  large  because  the  technology 
is  more  effective  and  it  is  also  cheaper.  This  ratio  is  a  conservative  estimate  because 
salvage  values  at  the  end  of  the  seven  year  life  span  were  not  considered.  Obviously,  the 
new  system  would  have  a  greater  salvage  value  than  the  old  system  at  the  end  of  its 
expected  life.  Therefore,  the  digital  imaging  technology  provides  more  than  twice  the 
benefit  of  the  film-based  system  and  is  also  cheaper. 
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not  a  significant  difference  in  the  answers  provided  in  the  questionnaire  from  the  two  sites. 
However,  a  trend  analysis  of  the  retake  data  provided  by  the  Madigan  Radiology 
Department  suggest  that  in  the  future  the  result  could  become  significant  as  technicians 
learn  to  use  the  digital  system  more  effectively. 

The  subsidiary  research  question  that  addresses  differences  in  the  number  of  lost 
images  favored  digital  imaging.  There  was  a  10%  difference  of  questionnaire  responses 
between  Madigan  and  Fitzsimons,  pertaining  to  lost  images,  that  favored  digital 
technology  verses  film.  The  qualitative  issues  associated  with  lost  images  including: 
incorrect  treatment,  delayed  diagnosis,  increased  exposure  as  a  result  of  retakes,  and 
decreased  productivity  in  clinics  make  digital  imaging  superior  to  film-based  imaging. 

The  third  subsidiary  question  addressed  time  to  retrieve  archived  images.  The 
results  of  the  questionnaire  clearly  show  that  physicians  can  retrieve  archived  images  faster 
under  a  digital  imaging  system.  It  remains  unclear  whether  this  time  saving  tool  improves 
physician  efficiency.  The  qualitative  issues  associated  with  this  MOE,  such  as  time  to 
diagnosis  and  treatment,  strongly  favor  digital  imaging. 

The  fourth  subsidiary  question  addresses  effectiveness  per  investment  dollar.  The 
denominator  in  this  approach  is  the  life  cycle  cost  of  the  system.  The  large  cost 
differences  between  the  two  systems  is  explained  by  two  primary  factors.  First,  the  new 
system  has  very  little  replacement  equipment  cost  in  the  seven  year  time  horizon  used  in 
the  modified  life-cycle  calculation.  Further,  the  MDIS  equipment  has  a  one  year  warranty 
period  built  into  the  contract,  during  which  there  are  virtually  no  maintenance  costs. 
Second,  the  older  system  requires  high  capital  investment  to  replace  old  equipment  and  to 
modernize  outdated  equipment  within  the  Radiology  Department.  The  fiscal  year  1993 
new  equipment  costs  at  Fitzsimons,  under  the  MEDCASE  program,  were  roughly  $5.2 
million.  However,  in  keeping  with  conservative  estimates,  only  a  $4.0  million  annual 
capital  equipment  cost  was  used  in  the  life-cycle  calculation. 
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D.  AREAS  FOR  FURTHER  RESEARCH 


There  are  many  opportunities  for  improving  the  Department  of  Defense  medical 
system  under  the  Government’s  National  Healthcare  Reform  initiatives.  The  Military 
Services,  in  a  joint  effort,  need  to  reengineer  clinical  practices  in  medicine,  with 
affordability  as  the  over-arching  goal  (Jenkins,  1995).  Seeking  technology  with  dual-use 
capability  between  civilian  and  military  healthcare  systems  decreases  development  and 
production  costs.  The  MDIS  system  is  a  step  toward  meeting  the  goals  of  these  initiatives. 
This  study  looked  at  whether  digital  imaging  improves  image  retake  rates,  decreases  lost 
images,  and  decreases  the  time  spent  retrieving  archived  images.  However,  there  are 
many  other  applications  associated  with  digital  imaging  that  are  yet  to  be  measured  in 
terms  of  cost,  benefits,  and  effectiveness.  When  all  phases  of  the  system  are  integrated 
within  and  between  facilities  these  applications  will  include: 

•  Computer-Aided  Diagnosis 

•  Real-time  Image  Interpretation 

•  Teleradiology 

•  Continuing  Medical  Education 

Researchers  at  the  University  of  Chicago  are  studying  computer-aided  diagnoses 
for  radiologists.  They  built  a  limited  database  of  patient  case  files  and  compared  the 
radiologist’s  diagnoses  with  computer  generated  diagnoses.  They  found  that  the 
computer  diagnosis  came  up  with  a  high  false  negative  rate.  But  when  the  computer 
provided  assistance  to  the  radiologist,  there  was  a  significant  decrease  in  the  number  of 
missed  diagnoses.  (Kunio,  1995)  Computer-aided  diagnosis  has  tremendous  potential  for 
improving  the  quality  of  patient  care,  but  cost  benefit  and  cost  effectiveness  has  yet  to  be 
determined. 

The  ideal  Radiology  Department  could  interpret  images  in  real-time.  This  time 
saving  technique  results  in  quicker  diagnosis  and  treatment  and  may  preclude  unnecessary 
return  visits  by  patients.  Some  improvements  to  the  current  system  must  be  implemented 
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before  this  can  be  accomplished.  One  example  is  increased  luminescence  to  achieve 
diagnostic  quality  images.  Radiology  Departments  may  be  able  to  reassign  radiologists  to 
specific  areas  (e.g.  Department  of  Surgery,  Orthopedics,  Emergency  Room)  to  facilitate 
real-time  image  interpretation. 

The  teleradiology  portion  or  phase  III  of  the  MDIS  implementation  schedule  will 
link  several  medical  treatment  facilities  from  all  Services  in  a  project  called  Seahawk.  The 
project  goals  include  increased  efficiency  through  teleconferencing,  a  corresponding 
decrease  in  patient  referrals  to  other  facilities,  and  improved  quality  of  patient  care  by 
providing  faster  Services. 

An  application  of  MDIS  is  developing  teaching  files  for  residents  and  continuing 
medical  education  for  physicians.  This  benefit  is  difficult  to  measure  in  terms  of  cost  but 
increases  the  quality  of  patient  care  which  can  be  quantified  in  future  studies. 

E.  SUMMARY 

The  MDIS  system  holds  promise  for  reengineering  healthcare  in  the  military.  The 
cost  effectiveness  of  the  system  is  demonstrated  in  this  as  well  as  previous  studies.  The 
MDIS  system  overcomes  many  of  the  limitations  of  film-based  technology,  including:  the 
ability  to  examine  an  image  at  one  location  at  a  time,  efficiently  transfer  images  from  one 
location  to  another  digitally,  and  overcoming  the  20-40%  loss  rate  experienced  in  many 
large  medical  treatment  facilities.  There  are  some  problems  with  the  technology  which 
also  must  be  overcome,  including  producing  a  diagnostic  quality  image  for  radiologists, 
establishing  an  interface  architecture  with  commercial  radiology  systems  such  as  magnetic 
resonance  imaging,  and  establishing  realistic  expectations  for  system  performance.  As  the 
technology  matures,  the  third-order  benefits  defined  by  Malone  and  Rockart,  1991  will 
evolve  making  the  system  more  effective  in  the  future. 
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APPENDIX  A 


IMAGING  SURVEY 


This  survey  evaluates  four  aspects  of  medical  imaging  in  your  facility. 

The  four  areas  to  be  assessed  are: 

•  Frequency  and  reasons  for  image  retakes 

•  Consequences  of  lost  images 

•  Reasons  and  frequency  of  lost  images 

•  The  time  to  retrieve  archival  images 

Please  take  a  moment  to  complete  this  survey.  Your  responses  will  be  used  to  evaluate  the  costs  and 
benefits  of  two  imaging  technologies. _ 


Circle  the  letter  which  best  answers  the  questions  below: 

1.  How  many  times  a  month  does  your  department,  ward,  or  clinic  request  archival  images  from  the  records 
storage  area  within  your  facility? 

a.  0-35 

b.  36-70 

c.  71-100 

d.  greater  100 

e.  don’t  know 

2.  On  the  average,  how  much  time  does  it  take  to  retrieve  an  image  from  your  organizations  record  storage 
location? 

a.  10  minutes 

b.  30  minutes 

c.  1-2  hours 

d.  more  than  2  hours 

e.  don’t  know 

3.  What  percent  of  archival  images  are  not  available  when  requested  by  a  clinician? 

a.  less  than  5% 

b.  5-10% 

c.  10-20% 

d.  greater  than  20% 

e.  don’t  know 


4.  What  percent  of  patient  images  require  retakes? 

a.  less  than  1% 

b.  1-5% 

c.  5-10% 

d.  greater  than  10% 

e.  don’t  know 


5.  Of  the  personnel  listed  below,  place  a  number  from  1  to  5  in  the  boxes  to  indicate  the  person  who  most 
frequently  retrieves  images  from  the  image  storage  room?  (1  indicates  most  frequent  and  a  5  indicates  least 
frequent.) 

Clerk|  |  Technician!  |  Nurse)  |  Physician|  |  Other  Q 

Please  take  a  moment  to  write  your  response  in  the  space  provided: 

6.  From  the  most  serious  to  least  serious,  list  the  5  most  severe  consequences  of  lost  images  in  your  department. 


(1) 

(2) 

(3) 

(4) 

(5) 


7.  From  the  most  frequent  to  least  frequent,  list  the  5  most  common  causes  of  lost  images  within  your  facility. 

(1)  _ _ _ 

(2)  _ _____ 

(3)  _ _ 

(4)  _ _____ 

(5) 

8.  Additional  Comments: 


Fill  in  the  appropriate  information: 

NAME  (Optional)  _ _ _ _ _ 

POSITION  _ _ _ 

RANK/GRADE  _ _ _ 

DEPT/CLINIC/WARD  _ __ _ 

Circle  your  response: 

Are  you  willing  to  discuss  this  survey  in  a  follow-on  interview?  YES  NO 

If  yes,  to  previous  question  please  provide  duty  phone  number  in  the  space  provided: 
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APPENDIX  B 


QUESTIONNAIRE  RESULTS 


Possible  resp. 


QUESTION  1 


CUM  PROB 


How  many  times  do  you  request  an  image  from  the  storage  area  within  your  facility? 

71-100  >100  Don‘tknow _ Possible  resp.  0-35  36-70  71-100 


QUESTION  1 


On  the  average,  how  much  time  does  it  take  to  retrieve  an  image  from  your  storage  location? 

1-2  hrs  >2  hrs  Don't  know _ Possible  resp.  10  min  30  min  1-2  hrs 


QUESTION  2 


What  percent  of  archival  images  are  not  available  (lost)  when  requested  by  a  clinician? 

1 0-20%  >20%  Don't  know _ Possble  resp. _ <5% _ 5-10%  10-20% 


QUESTION  6  From  the  most  serious  to  least  serious,  list  the  5  most 
severe  consequences  of  lost  images  in  your  department. 


Frequency 


MAMC  FAMC 


Delayed  diagnosis  and  treatment 


Retake  image 


Increase  time  and  money 


Patient  dissatisfaction 


Physician  frustration  _ 


Unable  to  compare  studies 


Death 


Increased  exposure 


Cancel/wrong  surgery 


Increase  hospital  stay 


Increase  clinic  wait 


Missed  diagnosis 


Wrong  treatment 


Poor  institutional  image 


QUESTION  7  From  the  most  frequent  to  least  frequent,  list  the  5  most 
common  causes  of  lost  images  within  your  facility _ 


Response  |  Frequency 


lack  hole,  the  $64,000  question 


MDIS  down 


Poor  film 


Physician  take 


Consultants  take 


Lost  in  MDIS 


Patients  Take 


Misfile 

8 

Clinic  takes 

1 

rays  not  read  by  radiologist 


Film  made  into  teaching  file 


Tech  does  not  put  on  system 


Not  ordered 


Doesn't  happen 


Confusion 


rds  take 


Lost  in  transit 


Mislabeled 
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QUESTION  8  Additional  Comments 


MAMC 

FAMC 

1 .  Most  of  the  images  are  on  computer  so  they  are  not  lost. 

2.  With  the  MDIS  system,  film  retrieval  is  far  easier  than 
facilities  without  MDIS. 

3.  You  should  define  your  terms  before  asking  questions. 

4.  The  system  (MDIS)  is  not  reliable. 

5.  Need  longer  times  before  station  signs  off  during  clinic  hours. 

6.  Most  of  the  time  there  are  no  serious  problems  with  MDIS 
since  retrieval  of  the  image  is  available  all  the  time. 

7.  The  only  way  to  get  the  important  studies  (CXR,  bone  films) 
is  to  ask  the  patient  to  bring  the  unread  wet  film  back,  and 
hand  carry  to  radiologist  for  diagnosis. 

8.  We  need  an  MDIS  terminal  in  oncology. 

9.  Still  patiently  awaiting  MDIS  for  HEME/Onc  section. 

1 0.  If  MDIS  is  the  answer  to  this,  we  are  extremely  frustrated 
by  still  not  having  a  single  MDIS  workstation  in  the  clinic 

1 .  1  really  would  like  to  retrieve  films  on  weekends,  is  this  possible? 

2.  Easy  access  to  timely  written  reports  would  be  more  helpful  than 
access  to  film. 

3.  Mortise  views  in  cast  are  most  common  cause  of  retakes. 

4.  Often  times  quality  of  images  is  not  adequate-  rotated  views 
accepted  by  technicians  require  reordering-  cause  clinic  delays. 
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The  following  bar  charts  show  a  comparison  of  each  medical  treatment  facility’s  questionnaire 
results. 


What  percent  of  patient  images  require  retakes? 


What  percent  of  archival  images  are  not  available  (lost)  when  requested  by  a  physician? 
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On  the  average,  how  much  time  does  it  take  to  retrieve  an  image  from  your  storage  location? 


Time  to  Retrieve  an  Archived  Image 


10  Mn.  30  Mn.  1-2  Hrs.  >2Hrs.  Unknown 

Time 


How  many  times  do  you  request  an  image  from  the  storage  area  within  your  facility? 


Number  of  Time  Physicians  Request  Images 
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APPENDIX  C 


INTERVIEW  INSTRUCTIONS 

My  name  is  Major  Tracey  Syvertson,  I  am  an  Army  Medical  Service  Corps  officer 
assigned  to  the  Naval  Postgraduate  School.  I  am  doing  a  study  which  compares  two 
different  imaging  technologies  at  Madigan  and  Fitzsimons  Army  Medical  Centers.  I 
would  like  to  ask  you  specific  questions  that  will  be  used  in  my  research  project.  Will  you 
assist  me  in  this  regard?  If  you  would  like  to  preserve  your  anonimity,  I  will  withhold 
your  name  from  the  study  and  quote  only  your  title. 
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